Skip to content

Conversation

stevensJourney
Copy link
Collaborator

@stevensJourney stevensJourney commented Nov 25, 2024

Overview

This moves the Mongo DB BucketStorage implementation out of the @powersync/service-core package to a new MongoDB Storage module.

This PR also aims to move all MongoDB specific functionality to the MongoDB module. This includes:

  • Migrations which are powered by MonogDB Migration stores.
  • MongoDB based exclusive locks.

Shared unit tests for the BucketStorageFactory are exported from the new @powersync/service-core-tests package. These tests are used for testing the MongoDB and Postgres bucket storage factories.

Rentacookie and others added 30 commits September 6, 2024 11:22
…rework-teardown

# Conflicts:
#	packages/service-core/src/routes/endpoints/admin.ts
…storage

Removed Postgres ZeroLSN from BucketStorage
…rework-teardown

# Conflicts:
#	packages/service-core/src/routes/endpoints/checkpointing.ts
# Conflicts:
#	modules/module-postgres/src/module/PostgresModule.ts
#	packages/service-core/src/api/diagnostics.ts
#	packages/service-core/src/replication/AbstractReplicationJob.ts
#	packages/service-core/src/replication/ReplicationModule.ts
#	packages/service-core/src/routes/endpoints/admin.ts
#	packages/service-core/src/routes/endpoints/sync-rules.ts
#	packages/service-core/src/system/ServiceContext.ts
#	packages/service-core/test/src/sync.test.ts
#	packages/types/src/config/PowerSyncConfig.ts
Implement powersync instance teardown functionality
…ture

# Conflicts:
#	packages/service-core/src/routes/endpoints/socket-route.ts
#	pnpm-lock.yaml
Copy link
Contributor

@rkistner rkistner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some initial comments on locks and migrations.

Copy link
Contributor

@rkistner rkistner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the structure here looks good to me.

One thing I'm wondering about in regards to the module structure - would it perhaps work better to create separate modules for replication vs storage?

One specific place this has an effect is with tests: Currently we have for example the mysql module with a devDependency on the mongodb module, so that the mongodb storage can be used with mysql tests. But we probably want to test all combinations of postgres/mongodb/mysql replication with mongodb/postgres storage. The dependency structure between the modules may start getting weird if we implement that.

@stevensJourney stevensJourney marked this pull request as ready for review January 8, 2025 09:44
@stevensJourney stevensJourney merged commit fea550f into main Jan 8, 2025
16 checks passed
@stevensJourney stevensJourney deleted the postgres-bucket-storage branch January 8, 2025 11:12
This was referenced Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants